Performance improvements to dual simplex: BFRT, hypersparsity, basis updates, primal infeasible list #192

chris-maes · 2025-07-08T00:05:37Z

This PR contains multiple performance improvements to the dual simplex code including:

O(N^2) -> O(N) bound-flipping ratio test
Exploiting hypersparsity (sparse factors plus sparse right-hand side) throughout the solver,
New middle-product form basis update (replaces Forrest-Tomlin since it currently handles hypersparsity better)
Maintaining and update a list of primal infeasibilites

New or existing test cover these changes. No changes in documentation are necessary.

…al error

…vement on NETLIB

Other experiments included in this PR: 1) Bound strengthing on CPU for dual simplex. Note that this did not lead to an improvement on the NETLIB LP test set. 2) Attempt at O(N) bound-flipping ratio test using bucket sort. This is stil a work in progress. 3) Attempt to compute Farkas certificate when no entering variable found in dual simplex ratio test. This has not been verified yet. Note that the maros NETLIB problem is classifed as infeasible with the list of primal infeasibilites and the updated pricing. As far as I can tell this is due to different choices for leaving variables in the pricing---that have the same score. To try to handle no entering variables better, I tried to remove the dual perturbation and reset the steepest edge. This allows me to continue on maros. But ultimately the problem is still classified as infeasible.

copy-pr-bot · 2025-07-08T00:05:40Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

…nd greenbeb as well. Farkas works on 17/24 NETLIB infeasible problems

chris-maes · 2025-07-23T22:07:56Z

/ok to test efb768c

chris-maes · 2025-07-23T23:02:26Z

/ok to test 3905665

KyleFromNVIDIA

Approved trivial CMake changes

…variable. Allocate timer to ftran

chris-maes · 2025-07-24T23:05:58Z

/ok to test 2230fbc

aliceb-nv

Thanks a lot for the awesome work Chris!
Just a few minor comments :)

aliceb-nv · 2025-07-28T15:05:25Z

cpp/src/dual_simplex/phase2.cpp

+
+  phase2::reset_basis_mark(basic_list, nonbasic_list, basic_mark, nonbasic_mark);
+
+  std::vector<bool> bounded_variables(n, false);


std::vector<bool> tends to be somewhat slower (as it's implemented through a bitmap), I'm not sure if this is anything close to a hotspot but for these use cases a std::vector or std::vector<uint8_t> work better

Ah yes. Good catch. The bitmap might slow down the access. I switched to uint8_t as suggested.

cpp/src/dual_simplex/phase2.cpp

aliceb-nv · 2025-07-28T16:10:02Z

cpp/src/dual_simplex/basis_updates.cpp

+  assert(row_permutation_.size() == m);
+  assert(rhs.n == m);
+  assert(solution.n == m);
+  assert(Lsol.n == m);


Those asserts will only be compiled when building in debug mode which we rarely ever do, we might want to use cuopt_assert instead (which is controlled by the "-a" flag in ./build.sh)

cpp/src/dual_simplex/basis_updates.cpp

aliceb-nv · 2025-07-28T16:40:37Z

cpp/src/dual_simplex/phase2.cpp

+                                   std::vector<i_t>& infeasibility_indices,
+                                   f_t& primal_inf)
+{
+  const f_t now_feasible = std::numeric_limits<f_t>::denorm_min();


Are denormals useful here? I think double should have enough precision, computations involving denormals usually imply a performance hit
As an aside: we might want to measure the performance/quality impact of flushing denormals to zero at some point for the CPU code, just in case

I'm not using the denormals for computation here. I was using this small value as a marker in the squared_infeasibilities array. If squared_infeasibilites[j] == now_feasible, than variable j has become feasible, and it can be removed from the infeasibility_indicies list. I think it is fine to just set the value to 0.0.

aliceb-nv · 2025-07-28T16:41:08Z

cpp/src/dual_simplex/phase2.cpp

+void clean_up_infeasibilities(std::vector<f_t>& squared_infeasibilities,
+                              std::vector<i_t>& infeasibility_indices)
+{
+  const f_t now_feasible = std::numeric_limits<f_t>::denorm_min();


Removed denormals. See above.

aliceb-nv · 2025-07-28T16:46:12Z

cpp/src/dual_simplex/phase2.cpp

+          std::vector<f_t> my_delta_y;
+          delta_y_sparse.to_dense(my_delta_y);
+
+          // TODO(CMM): Do I use the perturbed or unperturbed objective?


Has this TODO been addressed?

Unfortunately, no. I'm still not sure which to use here. Right now the objective value is just for printing in the logs. So it doesn't affect the solution.

aliceb-nv · 2025-07-28T16:46:47Z

cpp/src/dual_simplex/phase2.cpp

+    // TODO(CMM): Do I also need to update the objective due to the bound flips?
+    // TODO(CMM): I'm using the unperturbed objective here, should this be the perturbed objective?


Has this TODO been addressed?

Unfortunately, no. Similar to the above I'm not sure what to do here. Luckily, it only affects the objective in the logs.

aliceb-nv · 2025-07-28T16:51:40Z

cpp/src/dual_simplex/sparse_vector.cpp

+  for (i_t k = 0; k < i.size() - 1; ++k) {
+    if (i[k] > i[k + 1]) { printf("Sort error %d %d\n", i[k], i[k + 1]); }
+  }


Just another tiny FYI: std::is_sorted is useful for this (in asserts and similar)

Thanks. I switched to std::is_sorted.

Thanks Alice!

chris-maes · 2025-07-29T02:36:28Z

/ok to test 0c4459e

aliceb-nv

Approved! Thanks a lot for the great work Chris!

tmckayus · 2025-07-29T15:23:55Z

This is not critical but is highly, highly desirable from a perception basis

…to test cast

chris-maes · 2025-07-30T18:27:51Z

/ok to test a939cf1

chris-maes · 2025-07-30T22:32:10Z

/ok to test 458e1dd

chris-maes · 2025-07-30T22:49:24Z

/ok to test 9dc4329

…al errors

chris-maes · 2025-07-31T04:01:41Z

/ok to test 7144d51

chris-maes · 2025-07-31T14:18:05Z

/merge

chris-maes added 6 commits June 11, 2025 13:44

Fix accidental O(N^2) BFRT with O(N log N). 13% better on NETLIB

96179e7

Don't keep looking for small pivots. Can turn infeasible into numeric…

e5ae745

…al error

First stab at hypersparse B and B^T solve

80594ae

Hypersparsity with MPF update

7da8a98

Dynamically switch between sparse and hypersparse solves. 1.42X impro…

fbae0b6

…vement on NETLIB

chris-maes added improvement Improves an existing functionality non-breaking Introduces a non-breaking change labels Jul 8, 2025

chris-maes added this to the 25.08 milestone Jul 8, 2025

chris-maes self-assigned this Jul 8, 2025

rgsl888prabhu and others added 5 commits July 17, 2025 11:44

Merge branch 'branch-25.08' into hypersparsity

b029775

Fix incorrect infeasibility classification of maros. Helps greenbea a…

35c0f46

…nd greenbeb as well. Farkas works on 17/24 NETLIB infeasible problems

Clean up code

3f0ca9f

Move sparse_vector_t into seperate files

c672416

More cleanup

efb768c

chris-maes marked this pull request as ready for review July 23, 2025 22:07

chris-maes requested review from a team as code owners July 23, 2025 22:07

chris-maes requested review from akifcorduk, aliceb-nv and rgsl888prabhu July 23, 2025 22:07

Formatting

3905665

KyleFromNVIDIA approved these changes Jul 24, 2025

View reviewed changes

rgsl888prabhu approved these changes Jul 24, 2025

View reviewed changes

chris-maes added 2 commits July 24, 2025 09:46

Add support for sparse vector rhs input to sparse triangular solve

7e7d0de

Remove unused sparse triangle solve accepting column of CSC as rhs

7cfe1d5

chris-maes added 3 commits July 24, 2025 15:35

Drop small delta_y. Always recompute primal variables if no entering …

0ca65ea

…variable. Allocate timer to ftran

Keep farkas off

1ac7e1f

Formatting

2230fbc

aliceb-nv mentioned this pull request Jul 25, 2025

Replace accidental O(N^2) BFRT with O(N log N). 13% better on NETLIB #96

Closed

aliceb-nv reviewed Jul 28, 2025

View reviewed changes

rgsl888prabhu approved these changes Jul 28, 2025

View reviewed changes

chris-maes added 4 commits July 28, 2025 10:19

Drop small elements. Dont copy intermediate solutions if not needed

42265d6

Address comments/suggestions in code review

8eb349a

Thanks Alice!

Formatting

45f5a43

Merge branch 'branch-25.08' into hypersparsity2

0c4459e

aliceb-nv approved these changes Jul 29, 2025

View reviewed changes

chris-maes added 5 commits July 30, 2025 08:22

Use std::is_sorted

60c03bb

Catch an issue when primal step length would be inf. Add more leeway …

ee5b179

…to test cast

Formatting

3d54385

Remove debug

be60ae7

Merge branch 'branch-25.08' into hypersparsity2

a939cf1

Loosen tolerance on test again

458e1dd

Merge branch 'branch-25.08' into hypersparsity

9dc4329

chris-maes added 3 commits July 30, 2025 20:49

Fix bug with nonbasic in infeasible list. Try to recover from numeric…

c7a1b0e

…al errors

Formatting

8d15c2e

Merge branch 'branch-25.08' into hypersparsity2

7144d51

rapids-bot bot merged commit b965d80 into NVIDIA:branch-25.08 Jul 31, 2025
73 checks passed


		phase2::reset_basis_mark(basic_list, nonbasic_list, basic_mark, nonbasic_mark);

		std::vector<bool> bounded_variables(n, false);

		// TODO(CMM): Do I also need to update the objective due to the bound flips?
		// TODO(CMM): I'm using the unperturbed objective here, should this be the perturbed objective?

Performance improvements to dual simplex: BFRT, hypersparsity, basis updates, primal infeasible list #192

Performance improvements to dual simplex: BFRT, hypersparsity, basis updates, primal infeasible list #192

Uh oh!

Conversation

chris-maes commented Jul 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

copy-pr-bot bot commented Jul 8, 2025

Uh oh!

chris-maes commented Jul 23, 2025

Uh oh!

chris-maes commented Jul 23, 2025

Uh oh!

KyleFromNVIDIA left a comment

Choose a reason for hiding this comment

Uh oh!

chris-maes commented Jul 24, 2025

Uh oh!

aliceb-nv left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chris-maes commented Jul 29, 2025

Uh oh!

aliceb-nv left a comment

Choose a reason for hiding this comment

Uh oh!

tmckayus commented Jul 29, 2025

Uh oh!

chris-maes commented Jul 30, 2025

Uh oh!

chris-maes commented Jul 30, 2025

Uh oh!

chris-maes commented Jul 30, 2025

Uh oh!

chris-maes commented Jul 31, 2025

Uh oh!

chris-maes commented Jul 31, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

chris-maes commented Jul 8, 2025 •

edited

Loading